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Summary 


This  paper  discusses  the  testing  of  methods  for  generating 
uniform  numbers  in  a  computer — the  commonly  used  multiplicative 
and  mixed  congruential  generators  as  well  as  two  new  methods. 

Tests  proposed  here  are  more  stringent  than  those  usually  applied, 
because  the  usual  tests  for  randomness  have  passed  several  of  the 
commonly  used  procedures  which  subsequently  gave  poor  results  in 
actual  Monte  Carlo  calculations.  The  principal  difficulty  seems 
to  be  that  certain  simple  functions  of  n- tuples  of  uniform  random 
numbers  do  not  have  the  distribution  that  probability  theory  predicts. 

Two  alternative  generating  methods  are  described,  one  of  them 
using  a  table  of  uniform  numbers,  the  other  one  combining  two 
congruential  generators.  Both  of  these  methods  passed  the  tests, 
whereas  the  conventional  multiplicative  and  mixed  congruential 


methods  did  not. 


Introduction 


This  paper  reports  the  results  of  extensive  testing  of  random 
number  generators,  including  two  which  we  propose  as  improvements  on 
present  methods.  We  carried  out  this  program  of  testing  because  random 
numbers  generated  by  mixed  congruential  methods  gave  bad  results  in  a 
number  of  Monte  Carlo  calculations,  notably  those  involving  order 
statistics.  The  tests  suggest  that  mixed  generators  are  unsatisfactory 
and  that  generators  of  the  straight  multiplicative  type  are  better  but 
still  suspect.  For  this  reason,  we  propose  two  generators  which  are 
variations  on  familiar  ideas.  One  combines  two  congruential  generators 
and  the  other  uses  a  table  of  random  numbers.  Both  tested  out  very  well. 

By  a  random  number  generator,  we  mean  any  procedure  for  producing 
within  a  computer  a  sequence  of  numbers  U^U^,...  which  is  supposed 
to  represent  a  sequence  of  independent  uniform  random  variables.  Although 
the  are  supposed  to  be  random,  they  are  usually  generated  by  a 

strictly  deterministic  procedure.  Most  often  the  are  regarded  as 

integers,  and  is  computed  by 

(i)  Ui+1  =  aUi  +  c  mod  M, 

where  M  is  generally  taken  as  2n  for  an  n-bit  binary  machine  and 
10n  for  an  n-digit  decimal  machine.  The  random  number  generator  is 
often  called"raultiplicative?’  if  c  =  0  and  ’’mixed"  if  c  ^  0. 

Generators  of  this  type  have  been  used  and  favorably  reported  by  many 
1  people;  an  extensive  list  of  references  may  be  found  in  the  survey  article 
[3].  Nevertheless,  we  have  had  trouble  with  mixed  generators.  This  led 
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us  to  study  the  testing  of  random  numbers.  We  decided  that  the  tests 
commonly  applied  to  random  number  generators  are  not  of  much  value. 

The  tests  we  discuss  in  this  paper  are  more  stringent  than  the  common 
tests,  but  not  at  all  unreasonable. 

In  attempting  to  improve  on  the  congruential  generators,  we  tried 
combining  two  of  them.  This  gave  a  generator  which  seems  to  be  better 
than  either  of  the  two  congruential  generators,  but  it  has  the  disadvantage 
of  being  slower.  We  also  investigated  the  possibility  of  using  a  table 
of  random  numbers.  This  turns  out  to  be  practical  on  a  large  computer 
with  buffered  input;  in  fact,  for  such  a  machine  we  recommend  this 
method. 

1.  A  random  number  generator  using  a  table  of  random  numbers 

An  obvious  alternative  to  the  use  of  a  numerical  procedure  for 
generating  unifonn  random  numbers  is  to  use  3ome  sort  of  table  of  random 
numbers.  The  big  advantage  of  this  is  that  one  should  be  able  to  get  a 
table  which  has  none  of  the  undesirable  characteristics  of  the  numeri¬ 
cally  generated  sequences.  The  two  main  problems  with  this  approach  are 
the  time  needed  to  read  the  table  into  the  computer  and  the  possibility 
of  exhausting  the  table.  In  this  section  we  describe  a  random  number 
generator  which  uses  a  table  but  overcomes  these  two  problems. 

To  overcome  the  time  problem  we  use  a  computer  with  buffered  input. 

We  use  two  blocks  of  core  storage  to  hold  random  numbers,  refilling  one 
block  from  tape  while  using  numbers  from  the  other  block.  Thus,  to  get 
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be  considered  a  set  of  435  independent  random  bits.  The  whole 
sequence  of  random  bits  was  grouped  to  form  the  table  put  on  tape. 

The  number  146  and  485  were  chosen  to  give  an  efficient 
conversion  from  decimal  random  digits  to  random  bits.  It  happens 
that  10 ^  >  2^  >  .9991  x  10^.  Therefore,  the  conversion  is  99.9 
per  cent  efficient  and  obtains  almost  the  maximum  of  log^lO  bits  per 
digit. 

In  the  test  results  summarized  in  Section  4>  the  first  run  used  the 
numbers  in  the  table.  The  second  run  used  the  transformation: 

U»  =  (215  +  5)U  +  3  mod  227, 

and  the  third  run  used 

U'  =  (27  +  1)U  +  1  mod  227. 

2.  A  combination  of  two  conpruential  generators 

The  generator  described  in  Section  1  is  suitable  only  for  computers 
with  buffered  input,  and  requires  tape  handling,  which  may  be  inconvenient. 
Therefore,  it  is  still  desirable  to  have  some  sort  of  numerical  generator. 
One  can,  of  course,  use  a  congruential  generator,  but  our  experience  with 
these  has  led  us  to  look  for  generators  with  better  statistical  properties. 

Suppose  the  first  number  for  a  congruential  generator  (l)  is 
picked  at  random.  Then  the  sequence  may  be  considered  a 

sequence  of  random  variables.  Moreover,  each  will  be  uniform  on 

[0,1],  or  rather  on  the  set  of  numbers  in  [0,1]  which  can  be  represented 
exactly  in  the  computer.  However,  the  different  th  are  not  independent, 
and  it  turns  out  that  the  distribution  of  an  n-tuple  (U^,...,U  )  may 


be  quite  far  from  the  correct  distribution*  In  order  to  improve  the 

\ 

distribution  of  n-tuples,  we  propose  using  two  different  generators 
of  the  type  (1)  and  having  one  shuffle  the  sequence  produced  by  the 
other. 


For  a  test  program,  we  used  the  generators 

(3)  Uk+1  =  (217  +  3)Uk  mod  235, 

and 

(4)  Vk+1  =  (27  +  l)Vk  +  1  mod  235. 

We  let  Uq  =  1  and  Vq  =  0.  A  table  of  128  locations  in  core  was  filled 

1L 

with  the  numbers  U^, ••  Then  to  generate  Xk>  the  k  n  random 

number  to  be  used,  we  used  the  first  7  bits  of  Vk  as  an  index  to  get 
from  the  table.  The  location  of  in  the  table  was  refilled  with 
the  next  number  from  the  generator  (3). 

The  time  to  generate  a  random  number  by  this  methou  is  about  twice 
the  time  required  with  a  congruential  generator.  We  consider  that  it  is 
worth  suffering  this  time  penalty  in  order  to  get  a  sequence  of  numbers 
with  better  statistical  properties.  From  the  test  results  given  in 
Section  4>  we  conclude  that  this  generator  does  indeed  have  better  proper¬ 
ties  than  the  congruential  generators;  refer  again  to  the  table  and 
remarks  in  that  section. 

3.  Testing  random  numbers 


Various  procedures  have  been  proposed  for  testing  sequences  of 
random  numbers,  and  extensive  references  may  be  found  in  the  survey 
article  [3].  However,  most  of  the  proposed  tests  seem  to  have  little 
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relevance  to  the  actual  uses  of  random  number  sequences.  Our  experience 
has  been  that  a  sequence  may  pass  these  tests  and  yet  give  a  distribu¬ 
tion  of  some  simple  function  f(U^,...,U  )  of  several  uniform  random 
variables  which  is  far  from  the  correct  distribution.  The  results 
reported  in  Section  4  should  convince  the  reader  of  this. 

The  two  most  commonly  used  tests,  the  chi-square  test  on  the  distri¬ 
bution  of  the  tL  ,  and  the  serial  test,  are  quite  useful  for  weeding  out 
some  of  the  unsatisfactory  generators  from  the  outsetj  they  are,  in  a 
sense,  necessary,  but  certainly  not  sufficient.  Unfortunately,  as 
usually  applied,  they  are  of  very  limited  precision.  For  example,  in 
[l],  Hull,  Dobell,  and  Allard,  following  the  usual  practice,  divided 
the  unit  interval  into  only  ten  equal  subintervals.  In  effect,  this  only 
tests  the  first  decimal  digit  of  the  uniform  numbers,  and  one  digit 
accuracy  is  low,  even  for  Monte  Carlo  calculations. 

Another  limitation  of  these  tests  is  that  they  say  nothing  about 
the  behavior  of  n-tuples  (U^,...,Un)  for  n  >  2.  This  is  a  serious 
limitation,  for  it  may  happen  that  pairs  ^e^ave  well  while 

triples  (U^jU^,^)  behave  very  badly.  The  tests  on  the  multiplicative 
generators  illustrate  this. 

Our  philosophy  of  testing  is  to  test  n-tuples  (U^, ,,.,Un)  for 
as  many  values  of  n  as  possible  and  to  make  the  tests  for  each  n 
more  stringent  than  is  common.  Even  so,  we  still  consider  that  the  best 
practice  is  to  test  the  sequence  to  be  used  for  a  particular  calculation 
by  trying  it  on  a  similar  problem  for  which  the  answer  is  known. 
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The  tests  we  actually  made  were  x  tests  on  the  distribution  of 
the  random  numbers,  pairs  of  random  numbers,  triples  of  random  numbers, 
and  various  simple  functions  of  several  random  numbers.  The  idea  of 
testing  simple  functions  of  several  random  numbers  was  to  have  some 
sort  of  test  of  the  behavior  of  n-tuples  (U^,...,Un)  for  n  >  3  and 
to  have  tests  of  pairs  and  triples  which  would  be  in  some  ways  more 
stringent  than  the  basic  tests. 

Some  of  the  tests  were  based  on  the  distribution  of  the  maximum 
and  minimum  of  n  uniform  variables.  These  were  chosen  as  the  easiest 
of  the  order  statistics  of  n  uniform  variables;  our  previous  dissatis¬ 
faction  with  order  statistics  generated  by  some  of  the  congruential 
generators  led  us  to  include  these  in  our  routine  tests  of  any  generators. 

We  also  routinely  test  the  distribution  of  the  sum  of  2,  3,  and  4 
uniform  variables,  as  these  distributions  are  useful  in  certain  procedures 
for  generating  arbitrary  random  variables. 

All  the  tests  had  the  same  general  form.  A  sequence  of  n  variables 
Xi,X2,...  was  computed  from  the  sequence  of  uniform  numbers.  If  the 
uniform  numbers  were  actually  independent  uniform  random  variables,  the 
X^  would  be  independent  identically  distributed  random  variables,  perhaps 
multi-dimensional.  The  range  of  the  X^  was  divided  into  m  cells  of 
equal  probability  p  and  the  number  of  occurences  in  each  cell  counted. 

The  x^-statistic 


m  (k.-np) 


was  computed.  We  then  computed  the  corresponding  percentile,  i.e.  the 

probability  that  for  a  random  variable  with  the  hypothetical  distribution 
2 

the  value  of  x  would  exceed  the  observed  value.  The  percentiles  were 
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rounded  to  two  figures  except  from  some  very  small  values.  These  per¬ 
centiles  are  tabulated  in  Section  4*  Where  c  appears,  it  means  the 
percentile  was  less  than  *02  per  cent# 

All  the  tests  were  made  simultaneously  using  the  same  sequence 
U1,U2,##*  ^us  resu^'ts  the  v&ri°us  tests  are  not  independent. 
Moreover,  as  a  programming  convenience,  we  took  the  uniform  numbers  in 
sets  of  ten#  From  a  set  of  ten  we  obtained  five  pairs  for  the  pairs  test, 
three  triples  for  the  triples  test,  and  one  each  of  the  various  one-dimen¬ 
sional  variables.  Thus  from  a  set  of  ten  uniform  numbers,  only  one  pair 
is  used  for  the  test  of  the  maximum  of  two  uniforms,  one  triple  for  the 
maximum  of  three,  etc.  The  exact  details  of  this  are  given  below. 

Three  separate  runs  were  made  for  each  generator,  and  the  different 
runs  were  independent.  In  fact,  each  run  started  at  the  point  in  the 
sequence  U^,U?,...  where  the  last  left  off. 

We  conclude  this  section  by  summarizing  the  details  of  each  test. 

Uniformity.  This  was  included  for  the  sake  of  completeness.  The  unit 
interval  was  divided  into  100  equal  cells.  For  each  run  10,000  uniform 
numbers  were  used,  the  exact  sequence  being  U^,U^^,U0^,. . .  The  occur¬ 
ences  in  each  cell  were  counted,  and  the  yK  statistic  computed. 

Pairs .  Successive  pairs  of  uniform  numbers  were  taken  as  the  coordinates 
of  a  point  in  the  unit  square.  The  unit  square  was  divided  into  400 
equal  cells.  For  each  run  a  total  of  50,000  pairs  was  generated. 
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Triples.  Successive  triples  of  uniform  numbers  were  taken  as  the  coordi¬ 
nates  of  a  point  in  the  unit  cube.  However,  every  tenth  uniform  number 
was  skipped.  The  cube  was  divided  into  1000  equal  cells.  For  each  run  a 
total  of  30,000  triples  was  generated. 

Maximum  of  n.  This  test  was  made  for  n  =  2,  3,  4>  5,  10.  Let  F 
denote  the  distribution  function  of  the  maximum  of  n  independent 
uniform  random  variables.  Then  if  V^,Vp,...,Vn  are  independent  uniform 
random  variables,  F(nax(V^,Vp,...  ,Vn) )  is  uniform  on  [0 ,1]  •  We  calcu¬ 
lated  F(max(Vn  ,Vp,...,Vn))  for  n-tuples  from  the  sequence  IJ  Up,... 

The  unit  interval  was  divided  into  100  equal  sub-intervals.  For  each 
run,  we  used  10,000  n-tuples.  The  sequence  of  n-tuples  actually  used 
was  (U^,...,Un),  (Un,...,U10+n),... 

Minimum  of  n.  This  test  was  the  same  as  that  for  the  maximum  of  n 
except  that  we  took  the  minimum  of  n-tuples  (V^,...,Vn)  and  used  the 
corresponding  distribution  function. 

Sum  of  n.  Again  this  test  was  similar  to  that  for  the  maximum  of  n. 

We  tested  to  see  if  G(V,  +  *«*+V  )  was  uniform,  where  G  is  the  distribution 
function  of  the  sum  of  n  uniform  random  variables. 

4.  Test  results  and  General  Remarks 

Besides  the  generators  described  in  Sections  1  and  2,  we  tested 
live  congruential  generators.  For  mixed  generators,  we  chose 

(5)  Uk+1  =  101Uk  +  1  mod  1010,  0Q  =  0, 

and 

(6)  Uk+1  =  (27  +  l)Uk  +  1  mod  235,  U0  =  0. 
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These  have  been  favorably  reported  on  in  [l]  and  [2],  To  represent 
multiplicative  generators,  we  chose 

(7)  Uk+1  =  (l00003)Uk  mod  1010,  UQ  =  1, 

and 

(8)  Uk+1  =  (217  +  3)Uk  mod  235,  UQ  =  1. 

g 

These  multipliers  insure  the  maximum  possible  period  of  5  x  10  and 
33 

2  respectively.  They  follow  the  suggestion  in  [4]  that  the  multiplier 
be  close  to  where  M  is  the  modulus.  Finally,  we  tested  Lehmer's 
original  method  for  the  Eniac. 

(9)  Uk+1  =  23Uk  mod  108  +  1,  VQ  =  47594118. 

This  was  the  first  congruential  generator  proposed,  and  it  is  still 
widely  used. 

The  table  reports  the  results  of  33  tests  made  on  each  of  the  seven 
generators.  These  do  not  include  all  the  tests  actually  performed,  but 
the  results  tabulated  do  give  a  fair  picture  of  the  behavior  of  the 
various  generators.  The  quantity  given  in  the  table  is  the  percentile 
fur  the  appropriate  chi-square  distribution.  Thus,  the  33  values  for 
each  generator  should  behave  somewhat  like  a  set  of  33  numbers  chosen 
uniformly  from  the  unit  interval.  There  may  be  occasional  small  or  large 
values,  but  any  preponderance  of  such  values  must  be  suspect. 

On  the  basis  of  these  tests,  which  may  be  viewed  as  preliminary, 
but  certainly  indicative,  the  two  mixed  congruential  methods  and  Lehmer's 
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original  method  are  not  satisfactory!  the  two  multiplicative  methods  are 
suspect,  while  the  methods  based  on  a  stored  table  or  on  mixing  of  two 
congruential  generators  appear  to  be  satisfactory.  It  is  true  that  the 
results  of  the  pairs  test  for  the  storea-table  generator  might  be  viewed 
with  suspicion.  Note,  however,  that  for  neither  the  stored-table 
generator  nor  the  combination  generator  do  any  epsilons,  denoting  a 
percentile  less  than  .02  per  cent,  appear  in  the  table,.  Moreover  if 
the  table  of  random  numbers  used  in  the  stored  table  generator  should 
turn  out  to  be  unsatisfactory ,  one  can  always  replace  it  with  a  better 
table.  We  expect  that  an  excellent  table  of  random  numbers  can  be  pre¬ 
pared  by  usi'-g  a  sequence  of  congruential  generators  to  repeatedly 
shuffle  either  the  set  of  numbers  produced  by  a  congruential  generator 
or  one  of  the  standard  tables  of  random  digits. 

In  conclusion,  we  would  like  to  point  out  that  with  the  high 
speeds  of  modern  computers  it  is  no  longer  important  to  look  for  the 
fastest  possible  random  number  generators.  All  generators  are  reasonably 
fast,  even  the  combination  of  two  congruential  generators  takes  only 
56  microseconds  per  random  number  in  the  7094  computer.  Therefore, 
we  should  look  for  the  generator  which  produces  the  most  suitable  sequences  of 
random  numbers.  We  expect  that  these  will  turn  out  to  be  generators 
using  a  table  of  random  numbers. 
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The  tabled  value  is  the  percentile  for  the  appropriate  chi-square  distribution 
Where  e  appears,  it  means  the  percentile  was  less  than  *02  per  cent. 
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The  tabled  value  is  the  percentile  for  the  appropriate  chi-square  distribution. 
Where  e  appears,  it  means  the  percentile  was  less  than  .02  per  cent. 
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